Combined Bilateral Filter for Enhanced Real-time Upsampling of Depth Images

Highly accurate depth images at video frame rate are required in many areas of computer vision, such as 3D reconstruction, 3D video capturing or manufacturing. Nowadays low cost depth cameras, which deliver a high frame rate, are widely spread but suffer from a high level of noise and a low resolution. Thus, a sophisticated real time upsampling algorithm is strongly required. In this paper we propose a new sensor fusion approach called Combined Bilateral Filter (CBF) together with the new Depth Discontinuity Preservation (DDP) post processing, which combine the information of a depth and a color sensor. Thereby we especially focus on two drawbacks that are common in related algorithms namely texture copying and upsampling without depth discontinuity preservation. The output of our algorithm is a higher resolution depth image with essentially reduced noise, no aliasing effects, no texture copying and very sharply preserved edges. In a ground truth comparison our algorithm was able to reduce the mean error up to 73% within around 30ms. Furthermore, we compare our method against other state of the art algorithms and obtain superior results.